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Abstract 

A systematic exposition is given of the theory of invariant differential operators on a not 
necessarily reductive homogeneous space. This exposition is modelled on Helgason's 
treatment of the general reductive case and the special nonreductive case of the space of 
horocycles. As a final application the differential operators on (not a priori reductive) 
isotropic pseudo-Riemannian spaces are characterized. 
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1 Introduction 

Let G be a Lie group and H a closed subgroup. Let g and f) denote the corresponding Lie 
algebras. Suppose that the coset space G/H is reductive, i.e., there is a complementary 
subspace m to f) in g such that Adc{H)m C m. Let B){G/H) denote the algebra of G- 
invariant differential operators on G/H. The main facts about D(G/if) are summarized 
below (cf. HELGASON [3, Ch.III], [4, Cor. X.2.6, Theor. X.2.7], [6, §2]). 

Let D(G) be the algebra of left invariant differential operators on G, D/^(G) the subal- 
gebra of operators which are right invariant under H and S{q) the complexified symmetric 
algebra over g. Let A: S'(g) — > D(G) denote the symmetrization mapping, /(m) denotes the 
set of AdG'(-ff)-invariants in S'(m). Then 

D^(G) =D(G)[)nDj^(G)©A(/(m)). (1.1) 

Let tt: G ^ G/H be the natural mapping. Let C'^{G) consist of the G°°-functions on G 
which are right invariant under H. Write / := / o tt (/ G C'^{G/H)) and {Duf)"" := uf 
if G C^{G/H), u G ©h(G)). Then G B){G/H). 

Theorem 1.1 The mapping u ^ is an algebra homomorphism from ©//(G) onto 
BiG/H) with kernel ]I}{G)i)nBH{G). The mapping P ^ Dx{p): I {xn) ^^{G / H) is a linear 
bijection. 
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Theorem 1.1. is of basic importance for the analysis on symmetric spaces. In particular, 
it can be shown that 3(0 /H) is commutative if G/H is a pseudo-Riemannian symmetric 
space which admits a relatively invariant measure. In its most general form this result was 
proved by DUFLO [1] in an algebraic way. G. van Dijk kindly communicated a short analytic 
proof of Duflo's result to me (unpublished). In [1] DUFLO used generalizations of (1.1) and 
Theorem 1.1 to the case of homogeneous line bundles over G/H. These can be proved by 
only minor changes of Helgason's original proofs. 

There exist nonreductive coset spaces G/H for which 'D(G/H) is still commutative. For 
instance, let G be a connected real semisimple Lie group and let M and be the usual 
subgroups of G. Then G/MN is the space of horocycles and D(G/MiV) is commutative. In 
order to prove this, formula (1.1) and Theorem 1.1 have to be adapted to the nonreductive 
case. While HELGASON [5, §4], [6, §3] has done this in an ad hoc way for the special coset 
spaces under consideration, it is the purpose of the present note to give a more systematic 
exposition of the theory of I!i{G/H) for a not necessarily reductive coset space. 

Furthermore, following Duflo, the theory will be developed for invariant differential op- 
erators on homogeneous line bundles over G/H. As a final application we will characterize 
3{G/H) for isotropic pseudo-Riemannian symmetric spaces G/if without a priori knowledge 
that G/H is reductive. Throughout HELGASON [4] will be our standard reference. 

2 Development of the general theory 

Let G be a Lie group with Lie algebra q. For X e g define the vector field X on G by 

{Xf)(g) := 1/(5 exptX) 1,^0, / e C^(G), g e G. (2.1) 

Then the mapping X i— > X is an isomorphism from q onto the Lie algebra of left invariant 
vector fields on G. Throughout this section let Xi, . . . , Xn be a fixed basis of q. 

For a finite-dimensional real vector space V the symmetric algebra S{V) is defined as 
the algebra of all polynomials with complex coefficients on V*, the dual of V. Let S'^{V) 
respectively SmiV) (m = 0, 1, 2, . . . ) denote the space of homogeneous polynomials of degree 
m on V*, respectively of polynomials of degree < m on V* . Thus S"^{G) is spanned by the 
monomials X^^Xj^ . . . Xj^ (ii, . . . , i„ G {1, . . . , n}). 

Let D(G) be the algebra of left invariant differential operators on G with complex coef- 
ficients. For P e S{g) define an operator A(P) on G°°(G) by 

(A(P)/)(5):=P('^,---,^)/(^exp(iiXi + --- + i„X„))[^ , (2.2) 



=t„=Q 



where 

d d \ & 



P { ^,---,^] — ^ ^ — for P = Xj, • • • Xi . 

It is proved in [4, Prop. II.1.9 and p. 392] that: 

Proposition 2.1 The mapping P i— > A(P) is a linear bijection from S{g) onto D(G). It 
satisfies 

A(y'") = f"*, re 0; (2.3) 
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A(yi...yj = i^ V Fi,...,F^e5. (2.4) 

ml ^ — ' 

T/ie definition of X is independent of the choice of the basis of g. 

The mapping A is called symmetrization. The Lie algebra q is embedded as a subspace 
of Bi{G) under the mapping X ^ X. Any homomorphism from to g uniquely extends to 
a homomorphism from D(G) to 3{G) and any linear mapping from g to g uniquely extends 
to a homorphism from S{g) to S{q). In particular, for g & G, the automorphism Ad{g) of q 
uniquely extends to automorphisms of both S{q) and D(G) and 

X{Ad{g)P) = Ad{g)X{P), P e S{q), g G G. (2.5) 

For g,gieG, / e C°°(G), D e ©(G) write 

Then 

Ad(y)D = D e D(G), ^ e G. (2.6) 

Let H he a closed subgroup of G and let [) be the corresponding subalgebra. Let m 
be a subspace of g complementary to (). Let Xi, . . . , be a basis of ra and Xr+i, . . . , X„ 
a basis of I). Let % be a character of H, i.e. a continuous homomorphism from H to the 
multiphcative group C\{0}. Throughout this section, H, m, the basis and x will be assumed 
fixed. 

Let n: G ^ G / H he the canonical mapping. Write := 7r(e). Let 

G^JG) := {/ e G^{G) I /(^7/i) = f{g)x{h-'), g e G, h e H}. (2.7) 

Sometimes we will assume that x has an extension to a character on G. This assumption 
clearly holds if % = 1 on if, but it does not hold for general H and %. For instance, if 
G — SU{2) or SL{2, M) and H — SO{2) then nontrivial characters on H do not extend to 
characters on G. 

If X extends to a character on G then we define a linear bijection f ^ f: G°°{G/H) 
C%^{G) by 

~f{9)--=f{<9))x{9-'), g^G. (2.8) 

Lemma 2.2 Let P e S{m). If X{P)f = for all f E G^.^{G) then P = 0. 

Proof For each / e G°°{G/H) we can find F e G^^^{G) such that 

F(exp(iiXi H h ir^r)) = f{ex.p{tiXi H h i^X^) • 0) 

for {ti, . . . , tr) in some neighbourhood of (0, . . . , 0). Hence 

= (A(P)F)(e) = P (-^, ■ ■ ■ , J-) /(exp(tiXi + • • • + t,X,) ■ 

\Otl OZf J tl=--=tr=0 
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ior Sill f eC^iG/H), so P = 0. □ 

Let the differential of x also be denoted by x- Let f)*^ be the complexification of i). Let 

f)^ := {X + x{X) I X e f)C} C B{G). (2.9) 

Clearly, D/ = if / G C^,^{G) and D G f)^. Let B){G)i)^ be the linear span of all vw with 
V e D(G), w e f)^. Observe that, by Proposition 2.1, Yl . . . e A(5'„i(0)) for Fi, . . . , F„ e 

0. The following proposition was proved in [4, Lemma X.2.5] for x = 1. 

Proposition 2.3 There are the direct sum decompositions 

X{SM) = XiSm-iigm"" e X{Sm{m)) (2.10) 

and 

B){G) = ©(G)!)^ ® A(5(m)). (2.11) 
Proof First we prove by complete induction with respect to m that 

X{SM) C \{S^.M)V + X{Sm{m)). 
This clearly holds for m = 0. Suppose it is true for m < d. Let 

P^X't---Xf^\ di + --- + dn^d. 

If dr+1 h o?n = 0, then P e 5'd(m), so A(P) e A(5'd(m)). If rf^+i + h lin > then, by 

(2.4), A(P) is a hnear combination of certain elements Yi- ■ - Yd with Y^ e h for at least one 

1, so 

A(P) e A(>Sd_i(0))^c + A(,Sd-i(0)) c A(5d-i(0))^^ + A(5d-i(0)). 

Now apply the induction hypothesis. This yields (2.10) and (2.11) (use Proposition 2.1) 

except for the directness. 

To prove the directness of the sum (2.11), suppose that P G <S'(m) and A(P) € B){G)i)^. 
Then A(P)/ = for all / G C^^(G'), so P = by Lemma 2.2. □ 

Lemma 2.4 Let D G B{G). Then Df ^0 for all f G C^^^{G) if and only if D e B{G)i)^. 

Proof Apply Proposition 2.3 and Lemma 2.2. □ 
Let us define 

BH,x,mod(G) := {D G B{G) \ Ad{h)D -De D(G)f)^ for all h G H}. (2.12) 
This definition is motivated by the following lemma. 

Lemma 2.5 Let D G D(G). Then the following two statements are equivalent. 

(i) DeB>H,^,^od{G). 

(ii) feGWjG)^DfeG^JG). 
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Proof Let D e B{G). If f e C^,^{G), heH then 

(★) {Df)^(^^ = j)R{h)jR{h) ^ ^{h-^)DRi^') f . 

First assume (i). If / G C^^(G), he H, then (D^(^) - D)f = {Ad{h)D - D)f = 0, so 
combination with (★) yields (Df)^^^^ = x{h^^)Df , i.e., Df e C^^(G). Conversely, assume 
(ii). If / e C^^^{G), heH, then (Df)^^^^ = x{h'^)Df, so combination with (★) yields 
_ D)f = 0. Hence Ad(/i)D - D = -De D(G')^>^ by Lemma 2.4. □ 

From the preceding results the following theorem is now obvious. 
Theorem 2.6 

(a) D//^^^mod(G') is a suhalgehra of3{G). 

(b) D(G)I)'^ is a two-sided ideal in Dj/,x,mod(G). 

(c) There is the direct sum decomposition. 

^H,x,modiG) = B{G)i)^ © A(5(m)) n Bh,^,^,^{G). (2.13) 

(d) Define the mappings A and B by 

u u{mod Il){G)i)^) '^lcg=' (G)" 



Then A is a linear bijection and B is an algebra isomorphism onto. 

Define the mapping cr: g — > m by 

a(X + Y):^X, Xem,Yei). (2.14) 

Consider ^(m) as a subalgebra of S{q). Thus, if P G S{m) and heH, then Ad{h)P G S{q) 
and aoAd{h)P G >S'(m) are well-defined. By an application of (2.4) we see that, if Q G Smio), 
then 

X{aQ -Q)e X{Sm-i{g)) + B{G)i)^. (2.15) 

Define the algebra 

/mod(m) := {P G S{m) \ a o Ad{h)P - P for aU h G ii"}. (2.16) 



Lemma 2.7 Lei P G 5(m) st<c/i i/iat A(P) G roj/,x,mod(G). l^nte P = P"" + P^_^, where 
pm g ^m^^)^ p^_^ g 5^_i(m). T/ien G /„,od(m). 
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Proof A(Ad(/i)P -P)e B{G)i)^ by (2.12). Hence 

A(Ad(/i)P'" - P'") e A(5„_i(s)) +D(G')^>^. 

So 

A(a o Ad(/i)P- - P-) e A(>5^_i(0)) + D(G')f)>^ c A(>5^_i(m)) + D(G)^^ 
where we used (2.16) and (2.10). By directness of the decomposition (2.10): 

a o Ad(/i)P™ - P"^ e -S^-i(m). 

Hence a o Ad(/i)P"* — P"*, being homogeneous of degree m, is the zero polynomial. 

Proposition 2.8 If X{Imod{m)) C BH,x,mod{G) then 

A(/mod(m)) = A(>S(m)) n D^.^^.n^odlC) 

and the mapping 



D 



■ A(/mod(tn)) Dj?,x,mod(G) 



is a linear bijection. 



□ 



Proof Use complete induction with respect to the degree of P G 'S'(m) in order to prove 
that P e /mod(Tri) if X{P) ^ ^H,x,mod{G) (apply Lemma 2.7). The second implication in the 
proposition follows from Theorem 2.6(d). □ 

Suppose for the moment that x extends to a character on q and remember the mapping 
f f defined by (2.8). For u G ^H,x,mod{G) define an operator Du acting on C°°{G/ H) by 



(Djr-.^uf, feC°°{G/H). 



(2.17) 



Then supp(P'„/) C supp(/), hence, by Pcctrc's theorem (cf. for instance NARASIMHAN 
[7, §3.3]), Dy, is a differential operator on G/H. One easily shows that Dy, e 'D{G/H), the 
space of G-invariant differential operators on G/H. 

Theorem 2.9 Suppose that x eoctends to a character on G. Then the mapping 



u 



c 



is an algebraic isomorphism onto. 



Du-. 



c 



B{G/H) 



Proof Clearly, C is an isomorphism into. In order to prove the surjectivity let D e 
3{G/H). Then there is a polynomial P e >S'(m) such that 



{Df){g . 0) = P ( ^, • • ■ , J- ) figeMtiXi + ■■■ + trX,) ■ 



=tr=0 
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for all / e C°°{G/H) and for g — e. By the G-invariance of D this formula holds for all 
geG. By (2.8) and (2.2) this becomes 

xiDfr = X{P){xf), i.e., {Dfr={x-'X{P)ox){f). 

Clearly, x"^A(P) ox G D(G) and, by Lemma 2.5, we have X~^KP)°X^ ^H,xmodiG). Thus, 
by (2.17),L' = L>^-U(P)ox. ' □ 

Suppose now that the coset space G/H is reductive, i.e., m can be chosen such that 
Ad(/i)m C m for all /i e if. Prom now on assume that m is chosen in this way. Let 

D^(G) := {D e B{G) \ Ad{h)D = D for all h G H}, (2.18) 

/(m) := {P e S{m) \ Ad{h)P = P for all h e H}. (2.19) 

Then 

A(5(m)) nD^,;,,^od(G) = A(/(m)) c Bh{G). 

Hence (2.13) becomes 

Oi/,x,mod(G') = 0(G'){)>^ e A(/(m)). (2.20) 
We obtain from Theorems 2.6 and 2.9: 

Theorem 2.10 Let G/H be reductive. Then: 

(a) Bh{G) is a suhalgehra of3{G). 

(b) B(G)f)>^ n Bh{G) is a two-sided ideal in Bh{G). 

(c) There is a direct sum decomposition 

Bh{G) = D(G)f)^ n Bh{G) © A(/(m)). (2.21) 

(d) Define the mappings A,B and C (Conly if x extends to a character on G) by 



u ^ ii(modD(G'){)^nDjy(G')) ^« 



^ D 



C 



A(7(m)) ^ Djf(G')/(D(G')f)>^nDjj(G)) ^Djf(G') ^ ^D(G///). 

r/ien ^4 is a linear bisection and B and C are algebra isomorphisms onto. 

The case % = 1 of Theorem 2.10 can be found in HELGASON [4, Cor. X.2.6 and Theor. 
X.2.7]. See DUFLO [1] for the general case. 
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3 Application to D(G/A^) and ^{G/MN) 



Let G be a connected noncompact real semisimple Lie group. We remember some of the 
structure theory of G (cf. [3. Ch.VI]): 

00 : Lie algebra of G. 

Q : complexification of go- 

9 : Cartan involution of go, extended to automorphism of g. 

00 = '^o + Po^ corresponding Cartan decomposition of go. 

I^p^: mamximal abelian subspace of po, ^ the corcsponding analytic subgroup. 

f)o : maximal abelian subalgebra of go extending (jp^. 

^k) '■— ^0 ^0) f)e its complexification 

I) : complexification of f)o; this is a Cartan subalgebra of g. 

A : set of roots of g with respect to f); the roots are real on H)^^ + f)pg. 

Introduce compatible orderings on f)*^ and + t)po)*- 

A+ : set of positive roots. 
P+ : set of positive roots not vanishing on [)pQ. 
P_ : set of positive roots vanishing on f)pp. 
g" : root space in g of a e A. 

no := nflgo. 

N : analytic subgroup of G corresponding to tIq. 

M : centralizer of [)p„ in G, Mq its identity component. 

trio : Lie algebra of M. 

tn : complexification of tUo; then 

tn=^e+ 5^(g'^ + 0-"). (3.1) 

aeP- 

Proposition 3.1 The coset spaces G/MN and G/N are not reductive. 

Proof Suppose that G/MN is reductive. Then there is an adg(m + n)-invariant subspace r 
of g complementary to m + n. Let a e P+ and let X be a nonzero element of g". For if e f) 
write H = Wh + Yh + Zh with Wh e x, Yh e m, Zh e n. Then, for each H e i): 

a{H)X = [WH + YH + ZH,X] 

so 

a{H)X - [Yh, X] - [Zh, X] - [Wh, X] e r n (m + n), 

so 

[YH,X] + [ZH,X]^a{H)X. 

It follows from (3.1) that 

[YH,X] + [ZH,X]eJ29''- 

/3eA 
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Hence a{H)X ~ for all H e h, so a — 0. This is a contradiction. 

In the case G/N the proof is almost the same: take r adg(n)-invariant and complementary 
to n and Yh — 0- □ 



HELGASON [5, p. 676] states without proof that G/MN is not in general reductive. 

Let {q be the orthoplement of nio in to with respect to the KiUing form on Qq. In order 
to apply Proposition 2.8 and Theorem 2.9 to B{G/MN) and D(G/iV) we take [q + hpo 
respectively to + f)po ^ complementary subspaces of trio + Uq respectively Uq in Qq- Now we 
have 

/mod(t0 + [)po) = S%,), (3.2) 

/mod(eo + [)po) = ^(mo + [)pJ. (3.3) 

(3.2) is proved in HELGASON [5. Lemma 4.2] and by only slight modifications in this proof, 

(3.3) is obtained. It follows from Lemma 2.5 that 

A(5({)pJ) C DMiV,l,mod(G) 

and 

A'',!, mod 

since M centralizes [)po and mo + l)po normalizes Uo. Gonsider D(A) and D(Mo/l) as sub- 
algebras of ©(G). Then D(^) C DMiv,i,mod(G') and D(MoA) C Djv,i,mod(G). It follows by 
application of Proposition 2.8 and Theorem 2.9 that: 

Theorem 3.2 The mapping u i— > (cf. (2.17)) is an algebra isomorphism from D(A) onto 
B{G/MN) and from B{MoA) onto B{G/N). In particular, B){G/MN) is a commutative 
algebra. 

The statements about B){G/MN) are in HELGASON [5, Theorem 4.1]. FARAUT [2, 
p. 393] observes that Helgason's result can be extended to the context of pseudo-Riemannian 

symmetric spaces. 

A special case of Theorem 6.2 can be formulated in the situation that G is a connected 
complex semisimple Lie group. Let g be its (complex) Lie algebra and put: 

u : compact real form of g. 

a : maximal abelian subalgebra of u. 

i) := a + ia; this is Gartan subalgebra of g. 

A : set of roots of q with respect to [). 

A+ : set of positive roots with respect to some ordering. 

: root space of o; e A. 
n := X]aGA+ fl"' ^ corresponding analytic subgroup, 
g'^ := considered as real Lie algebra. 
if' := I) considered as real subalgebra. 

Then g'* = u + ia + n is an Iwasawa decomposition for (cf. [4, Theorem VI. 6. 3]) and a is 
the ccntralizer of ia in u. Hence we obtain from Theorem 3.2: 

Theorem 3.3 The mapping P i— >• i^A(P) is an algebra isomorphism from S{\:)^) onto 
B>{G/N). In particular, D(G/A^) is commutative. 

This theorem was proved by HELGASON [6, Lemma 3.3] without use of Theorem 3.2. 
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4 Application to isotropic spaces 

We preserve the notation and conventions of Section 2. First we prove an extension of [4, 
Cor. X.2.8] to the case that G/ H is not necessarily reductive. In the following, A and B are 
as in Theorem 2.6(d). 

Lemma 4.1 // the algebra /mod(iTi) is generated by Pi, . . . , Pi and if there are Qi, ■ ■ ■ ,Qi € 
Sjn such that degree(Pi — Qi) < degree Pj and \{Qi) G '^H,x,moA{G) then the algebra 



Di?,X,mod 



is generated by BAX{Qi), . . . , BAX{Qi). 



Proof We prove by complete induction with respect to m that, for each P G Sm{m) 
with A(P) e ^H,x,mod{G): BAX{P) depends polynomially on BAX{Qi),...,BAX{Qi). In 
view of Theorem 2.6 this will prove the lemma. Suppose the above property holds up to 
m — 1. Let P e ^'^(Tn) such that A(P) e ^H,x,inod{G). By using Lemma 2.7 we find that 
P = n(Pi,...,P;) (mod S'm-i(tTi.)) for some polynomial 11 in I indeterminates. Hence, 
P^n{Qi,...,Qi) (mod 5^_i(m)), 

A(P) = X{U{Qi,...,Qi)) (mod A(5'™_i(m))) 

= n(A(gi),...,A(QO) (mod A(5„_i(0))), 
A(P) - n(A(Qi), . . . , X{Qi)) e X{S^_i{q)) n BH,x,mod{G). 

By Theorem 2.6 and formula (2.10) we have 

BAX{P) - U{BAX{Qi), BAX{Qi)) = BAX{P') 

for some P' G 5'^_i(m) such that A(P') e '^H,x,m.oA{G). Now apply the induction hypothesis. 

□ 

Let T denote the action of G on G/H. Its differential dr yields an action of H on the 
tangent space {G/H)q to G/H at 0. 

Theorem 4.2 Suppose there is a nondegenerate dr{H) -invariant bilinear form (■, ■) on 
{G/H)q of signature (ri,r2) (ri + r2 = r, ri > such that, for each A > 0, dr^H) 
acts transitively on {X e {G/H)q \ {X, X) = A} (or on the connected components of these 
hyperbolas if ri — r2 — 1). Let A be the Laplace- Beltrami operator on G/H coresponding 
to the G -invariant pseudo-Riemannian structure on G/H associated with (•,•). Then the 
algebra B){G/H) is generated by A, and hence commutative. 



Proof Choose a complementary subspace m to [) in q. The mapping dTi identifies the H- 
spaces m (under a o AdoiH)) and {G/H)o (under dT{H)) with each other. Transplant the 
form (-, ■) to m and choose an orthonormal basis Xi, . . . , of m: (Xj, Xj) = EiSij, Si = 1 
or —1 for i < ri or > ri, respectively. Then the algebra /mod(Tn) is generated by J^^^^ ^v^l- 
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It follows from the proof of Theorem 2.9 that A = -Da(p) with P e >S'(m) of degree 2 such 
that \{P) e ^H,i,mod{G). Thus, by Lemma 2.7, we get 

J, 

P = c^eiXl (mod^iH) 

i=l 

with c 7^ 0. Now apply Lemma 4.1 and Theorem 2.9. □ 

Theorem 4.2 extends [4, Prop. X.2.10], where the case is considered that G/H is a 
Riemannian symmetric space of rank 1. A pseudo-Ricmannian manifold M is called isotropic 
if for each x E M and for tangent vectors X,Y ^ at x with {X, X) = {Y, Y) there is 
an isometry of M fixing x which sends X to Y. Connected isotropic spaces can be written 
as homogeneous spaces G/H satisfying the conditions of Theorem 4.2 with G being the 
full isometry group (cf. WOLF [8, Lemma 11.6.6]). It follows from Wolf's classification 
[8, Theorem 12.4.5] that such spaces are symmetric and reductive. However, our proof of 
Theorem 4.2 does not use this fact. 
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